Skip to content

Fix pre-weights ignoring dit_quant_scheme (fp8 dtype mismatch) - #1470

Open
li-lizhe wants to merge 1 commit into
ModelTC:mainfrom
li-lizhe:fix-pre-weights-dit-quant-scheme
Open

Fix pre-weights ignoring dit_quant_scheme (fp8 dtype mismatch)#1470
li-lizhe wants to merge 1 commit into
ModelTC:mainfrom
li-lizhe:fix-pre-weights-dit-quant-scheme

Conversation

@li-lizhe

@li-lizhe li-lizhe commented Sep 1, 2026

Copy link
Copy Markdown

Problem

Running a pre-quantized FP8 checkpoint (e.g. qwen_image_edit_2509_fp8_e4m3fn_scaled.safetensors) with dit_quant_scheme: fp8-sgl crashes on the very first matmul:

File "lightx2v/common/ops/mm/mm_weight.py", line 344, in apply
    return torch.addmm(self._get_actual_bias(), input_tensor, self._get_actual_weight(), out=output_tensor)
RuntimeError: self and mat2 must have the same dtype, but got BFloat16 and Float8_e4m3fn

Root cause

ZImagePreWeights hard-codes its matmul layers (img_in / txt_in / timestep embedder) to the "Default" scheme:

self.add_module("img_in", MM_WEIGHT_REGISTER["Default"](...))

while ZImagePostWeights and ZImageTransformerWeights both resolve config["dit_quant_scheme"] and register through MM_WEIGHT_REGISTER[self.mm_type].

So with a quantized checkpoint, all_x_embedder (and the other pre layers) load their float8_e4m3fn weights but run the Default torch.addmm path — a bf16 activation multiplied against an fp8 weight.

Fix

Align ZImagePreWeights with the other two weights classes: resolve mm_type once and register every matmul layer through MM_WEIGHT_REGISTER[self.mm_type]. The x_pad_token / cap_pad_token tensor registrations are left untouched.

Verification

  • python -m py_compile passes.
  • AST assertion confirms all four ZImagePreWeights matmul layers now follow self.mm_type, matching ZImagePostWeights (which was already correct).
  • This is a scheme-dispatch consistency fix; the FP8 kernel path itself (_scaled_mm / cutlass) is CUDA-specific, so I could not run end-to-end FP8 inference here. The change simply routes the pre-embedding matmuls through the same quantized/mm dispatch the maintainer already uses for post/transformer weights, which resolves the dtype mismatch on the reported setup.

Fixes #1439

ZImagePreWeights hard-coded its matmul layers (img_in / txt_in /
timestep embedder) to the "Default" scheme, unlike ZImagePostWeights
and ZImageTransformerWeights which follow config["dit_quant_scheme"].

With a pre-quantized FP8 checkpoint (e.g.
qwen_image_edit_2509_fp8_e4m3fn_scaled.safetensors), all_x_embedder is
stored as float8_e4m3fn but img_in still runs the Default addmm path,
so the bf16 activation and the fp8 weight fail with:

    RuntimeError: self and mat2 must have the same dtype,
    but got BFloat16 and Float8_e4m3fn

Align pre-weights with post/transformer weights: resolve mm_type once
from config and register every matmul layer via
MM_WEIGHT_REGISTER[self.mm_type].

Fixes ModelTC#1439
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] Qwen-Image-Edit-2509 Scaled FP8 inference fails with BF16/FP8 dtype mismatch on RTX A6000

1 participant